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Abstract 

Resource scheduling in infrastructure as a service (laaS) is one of the keys for large-scale Cloud 
applications. Extensive research on all issues in real environment is extremely difficult because it requires 
developers to consider network infrastructure and the environment, which may be beyond the control. In 
addition, the network conditions cannot be controlled or predicted. Performance evaluations of workload 
models and Cloud provisioning algorithms in a repeatable manner under different configurations are 
difficult. Therefore, simulators are developed. To understand and apply better the state-of-the-art of 
cloud computing simulators, and to improve them, we study four known open-source simulators. They 
are compared in terms of architecture, modeling elements, simulation process, performance metrics and 
scalability in performance. Finally, a few challenging issues as future research trends are outlined. 
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1. Introduction 

Cloud computing is developed based on vari¬ 
ous recent advancements in virtualization. Grid 
computing, Web computing, utility computing 
and related technologies. Cloud computing pro¬ 
vides both platforms and applications on demand 
through the Internet or intranet [19] . Some of the 
key benefits of Cloud computing include the hid¬ 
ing and abstraction of complexity, virtualized re¬ 
sources and efficient use of distributed resources. 
Some examples of emerging Cloud computing 
platforms are Google App Engine [T5|, IBM blue 
Cloud [16], Amazon EC2 [6], and Microsoft Azure 
m- Cloud computing allows the sharing, al¬ 
location and aggregation of software, computa¬ 
tional and storage network resources on demand. 
Cloud computing is still considered in its infancy 
as there are many challenging issues to be resolved 
[EKBESl. Youseff et al. m establish a detailed 
ontology of dissecting Cloud into five main lay¬ 
ers from top to down: Cloud application (SaaS), 
Cloud software environment (PaaS), Cloud soft¬ 
ware infrastructure (laaS), software kernel and 
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hardware (HaaS), and illustrate their interrela¬ 
tions as well as their inter-dependency on preced¬ 
ing technologies. 

Cloud data center can be a distributed network 
in structure, which is composed of many comput¬ 
ing nodes (such as servers), storage nodes, and 
network devices. Each node is formed by a se¬ 
ries of resources such as CPU, memory, network 
bandwidth and so on. Each resource has its cor¬ 
responding properties. There are many different 
types of resources for Cloud providers. The defi¬ 
nition and model defined by this paper are aimed 
to be general enough to be used by a variety of 
Cloud providers. In this paper, we focus on Infras¬ 
tructure as a service (laaS) in Cloud data centers. 

In a traditional data center, applications are 
tied to specific physical servers that are often 
over-provisioned to deal with workload surges 
and unexpected failures [5]. Such configuration 
rigidity makes data centers expensive to main¬ 
tain with wasted energy and floor space, low re¬ 
source utilizations and significant management 
overheads. With virtualization technology, to¬ 
day’s Cloud data centers become more flexible, 
secure and on-demand allocating. 

One key technology plays an important role in 
Cloud data center is resource scheduling. One 
of the challenging scheduling problems in Cloud 
data center is to consider allocation and migration 
of reconflgurable virtual machines and integrated 
features of hosting physical machines. 
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It is extremely difficult to research widely for 
all these problems in real platforms because the 
application developers can’t control and process 
network environment. What is more, the network 
conditions cannot be predicted or controlled. 

The research of dynamic and large-scale dis¬ 
tributed environment can be achieved by build¬ 
ing data center simulation system, which supports 
visualized modeling and simulation in large-scale 
applications in cloud infrastructure. Data cen¬ 
ter simulation system can describe the application 
workload statement, which includes user informa¬ 
tion, data center position, the amount of users 
and data centers, and the amount of resources in 
each data center. Using this information, data 
center simulation system generates requests and 
allocates these requests to virtual machines. 

By using data center simulation system, ap¬ 
plication developers can evaluate suitable strate¬ 
gies such as distributing reasonable data cen¬ 
ter resources, selecting data center to match 
special requirements, improving resource utiliza¬ 
tion and load balancing, reducing total energy- 
consumptions, reducing costs and so on. We will 
look at some closely related work firstly. 

1.1. Related Work 

There is quite intensive research conducted for 
cloud simulators. In this paper, we concen¬ 
trate on open-source simulators which we can eas¬ 
ier access. Dumitrescu and Foster [8] introduce 
GangSim tool for grid scheduling. Buyya et al. 
introduce GridSim m toolkit for modeling and 
simulation of distributed resource management 
for grid computing. Galheiros et al. [25] intro¬ 
duce modeling and simulations of Cloud comput¬ 
ing environments at application level, a few sim¬ 
ple scheduling algorithms such as time-shared and 
space-shared are discussed and compared. Sakel- 
lari et al. M complement a survey of mathemat¬ 
ical models, simulation approaches and testbeds 
in cloud computing, which aims to enable re¬ 
searcher to find suitable modelling approach and 
simulation implementation. Ikram et al. [2] in¬ 
troduce a novel cloud resource management ser¬ 
vice model and its simulation-based evaluations 
are mainly focusing on two applications dynamic 
service composition. Nuu et al. [27| propose 
a scheme for modeling and experimenting com¬ 
bined smart sleep and power scaling algorithms 
in energy-aware data center networks. Gurout et 
al. [26] provide a survey on energy-aware sim¬ 
ulation techniques with DVFS (Dynamic Voltage 
and Frequency Scaling). CloudAnalyst [7] aims to 
achieve the optimal scheduling among user groups 
and data centers based on the current configura¬ 
tion. Both CloudSim and GloudAnalyst are based 


on Sim Java m and GridSim m, which treat a 
Cloud data center as a large resource pool and 
consider application-level workloads. Kliazovich 
et al. m propose an energy-aware simulation en¬ 
vironment named GreenCloud for Cloud datacen¬ 
ters at package level. Nunez et al. [4] introduce 
a new simulator of cloud infrastructure named 
iCanCloud using C++ and compare the perfor¬ 
mance with CloudSim. Tian et al. [32] propose 
CloudSched, a novel lightweight simulation tool 
for VM scheduling with lifecycle in Cloud data 
centers. 

1.2. Comparative Guideline of Open-Souree 
Cloud Simulators 

Cloud simulators can be divided into various 
categories according to their features. In this sec¬ 
tion, we will give a brief comparison with different 
categories by extending the comparison category 
in [9]. The open-source simulators are selected be¬ 
cause we can study their source codes in details, 
develop new algorithms and improve them if nec¬ 
essary. The four open source simulators, namely 
CloudSim, iCanCloud, GreenCloud, CloudSched, 
are representative of many related simulators be¬ 
cause we study the architecture design, modeling 
elements, simulation process, performance met¬ 
rics and scalability. These simulators have com¬ 
mon features such as in architecture, modeling 
elements, simulation process as well as their own 
characteristics such as focusing on different ser¬ 
vice layers and with different performance met¬ 
rics. CloudSim is well known simulator for cloud 
computing, it can be extended easily but cur¬ 
rently it does not consider parallel experiments 
or lifecycles of VMs. The iCanCloud implements 
parallel experiments but does not consider en¬ 
ergy consumption or VM migration. GreenCloud 
models detailed energy consumptions for differ¬ 
ent physical components. CloudSched can model 
lifecycle of requests, and provide different metrics 
for load-balance, energy efficiency and utilization 
etc. Four open source cloud data centers simula¬ 
tors (CloudSim, GreenCloud, iCanCloud, Cloud¬ 
Sched) are compared together in Table 1. 

Platform: The platform that the simulator 
based on makes it bind with some specific fea¬ 
tures. CloudSim and CloudSched are both im¬ 
plemented with Java, so they can be executed on 
any machine installed JVM. While built based on 
GridSim and Sim Java, CloudSim is heavy to exe¬ 
cute. GreenCloud is an extension of NS2 network 
simulator, and it’s a packet level simulator. As 
for iCanCloud, it’s based on OMNET, which can 
simulate in-depth physical layer entities. 

Language: The languages implemented the 
simulators are related to the platforms. CloudSim 
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Table 1: Comparison Guideline 


Items 

CloudSim [24] 

GreenCloud [9] 

iCanCloud [3] 

CloudSched [30] 

Platform 

any 

NS2 

OMNET, MPI 

any 

Programming Language 

Java 

C++/OTcl 

C++ 

Java 

Availability 

Open Source 

Open Source 

Open Source 

Open Source 

Graphical Support 

Limited (Via CloudAnalyst) 

N 

N 

Y 

Physical Models 

N 

Limited (Via Plug-in) 

Y 

Y 

Models for public cloud 

N 

N 

Y 

Y 

Parallel experiments 

N 

N 

Y 

N 

Energy Consumption 

Y 

Y 

N 

Y 

Migration algorithms 

Y 

N 

N 

Y 

Simulation time 

Seconds 

Tens of minutes 

seconds 

seconds 

Memory space 

small 

large 

medium 

small 


and CloudSched are implemented with Java, 
while GreenCloud needs combining C+H- and 
OTcl, iCanCloud is in C+H-. 

Availability: The four simulators under dis¬ 
cussion are free or open-source, available for pub¬ 
lic download. 

Graphical support: The original CloudSim 
supports no graphical interface, the graphical in¬ 
terface is supported in Cloud Analyst. However, 
full support is not provided in Cloud Analyst, only 
the configurations and results can be presented. 
So we label it as limited, the same reason is 
also applicable for CreenCloud. CloudSched and 
iCanCloud support whole scheduling process to 
be showed on the interfaces. 

Physical server models: The details about 
the simulated components can reflect the preci¬ 
sion of the simulator and the validity of the re¬ 
sults. iCanCloud and CloudSched provide de¬ 
tailed simulation for physical analogs for the 
scheduling, which can trace resource utilization 
in physical servers and rejected requests informa¬ 
tion. CreenCloud needs to use a plug-in to simu¬ 
late and then it can even capture the packet loss. 
CloudSim treats resource pool as a whole. 

Models for public cloud providers: Ama¬ 
zon, as a cloud provider, has proposed its VM 
models and informed that by using these specifica¬ 
tions, better scheduling effects could be obtained. 
Both iCanCloud and CloudSched use the model 
suggested by Amazon, in which physical machine 
and virtual machine specifications are pre-defined. 

Parallel experiments: Parallel experiments 
could combine more than one machine to work 
together to process the tasks. Supporting for mul¬ 
tiple machines running experiments together is a 
main feature of iCanCloud and that feature is not 
presented in other three simulators. 

Energy consumption model: The energy 
consumption model can enable the simulators to 
compare energy efficiency of different scheduling 
strategies and algorithms. Except for iCanCloud, 
other three simulators can support energy con¬ 


sumption modeling. The energy consumption 
model implemented in CreenCloud can trace ev¬ 
ery element in a data center. DVFS energy con¬ 
sumption model is proposed in CloudSim with ex¬ 
tension tools. CloudSched provides energy con¬ 
sumption metrics for different scheduling algo¬ 
rithms. 

Migration algorithms: Migration algorithms 
are proposed to satisfy specific objectives, for in¬ 
stance dealing with the overloaded scenario in 
load balancing applications, reducing the total 
number of running machines to save total energy 
consumption, improving the resource utilization 
and so on. CloudSim and CloudSched support 
migration algorithms, while other two simulators 
do not. 

Scalability: This mainly means how fast the 
simulator can run (simulation time) and how 
much memory space the simulator will consume 
as the total number of requests is increasing, es¬ 
pecially to a large amount. We will provide com¬ 
parison in performance evaluation. 

In summary, CloudSim, CreenCloud, iCan¬ 
Cloud and CloudSched are open source and avail¬ 
able to download. CloudSim and GreenCloud of¬ 
fer no graphical interface support; CloudSched 
and iCanCloud all provide user interface to oper¬ 
ate. CloudSched and iCanCloud support physical 
server models, and GreenCloud supports physical 
models with a plug-in. In addition, CloudSched 
and iCanCloud offer models for public cloud 
providers. Parallel experiments are supported 
only in iCanCloud, but only iCanCloud does not 
support energy consumption model. CloudSim 
and CloudSched implement migration algorithms 
while others not. In the following sections, we 
will provide in-depth comparative study in terms 
of architecture design, simulation process, ele¬ 
ments, performance metrics and scalability in per¬ 
formance. 

The organization of remaining parts of this pa¬ 
per is as follows: from section 2 to section 6, 
detailed comparisons from different views about 
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CloudSim, GreenCloud, iCanCloud and Cloud- 
Sched are given. Section 2 compares the architec¬ 
ture and main features of these simulators; sec¬ 
tion 3 compares the way how elements are mod¬ 
eled in different simulators; section 4 presents the 
basic simulation process and compares minor dif¬ 
ferences in those simulators; section 5 lists the 
metrics in use; section 6 shows how performance 
are evaluated in those simulators; finally conclu¬ 
sions about cloud simulators are given. 

2. Comparison 1: Architecture and Main 
Features 


In this section, we will discuss the simulators 
architectures. 



Figure 1: The Architecture of CloudSim |25| 


Fig. 1 shows the multi-layered design and im¬ 
plementation of CloudSim. At the fundamental 
layer, management of applications, hosts of VMs, 
and dynamic system states are provided. By ex¬ 
tending the core VM provisioning functionality, 
the Cloud provider can also study the efficiency 
of different strategies at this layer. As for the 
top layer, the User Code represents the basic en¬ 
tities for hosts, and through extending entities at 
this layer, developer can enable the application to 
generate requests in a variety of approaches and 
configurations, model cloud scenarios, implement 
custom applications, and etc. 

CreenCloud structure could be mapped on the 
three-tier data center architectures as in Fig. 2, 
which are the most common architectures. Basi¬ 
cally, the architectures are composed of access lay¬ 
ers, aggregation layers and cores layers. Servers 
are placed at the access layer and responsible for 
task execution. Switches and Links form the in¬ 
terconnection fabric that delivers workload to any 
of the computing servers for execution at the ag¬ 
gregation layer. The core layer constitutes the 



Figure 2: Three-tier data center architecture of 

GreenCloud |10| 


workloads that can model various cloud user ser¬ 
vices. 



Figure 3: The Architecture of iCanCloud [4] 


The iCanCloud adopts the architecture shown 
in Fig. 3, which is also a layered architecture. The 
bottom of the architecture consists of the hard¬ 
ware models layer, which basically contains the 
models that are in charge of modeling the hard¬ 
ware parts of a system. A set of system calls are 
connected with the hardware models layer in the 
basic systems API module. In this module, a set 
of system calls are provided as Application Pro¬ 
gramming Interface for all applications run in a 
VM. The upper layer is a VMs repository, which 
contains a collection of VMs previously defined 
by the user. The cloud hypervisor is at the up¬ 
per layer that is managing all produced jobs and 
the instances of VMs where those jobs are exe¬ 
cuted. As for the top of architecture, it contains 
a definition of the entire cloud system. 

CloudSched is implemented under a simplified 
layered architecture as shown in Fig. 4. From 
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Figure 4: A Simplified Layered Architecture of CloudSched 
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top to bottom layer, at the top layer, there is an 
interface for a user to select resources and send re¬ 
quests, basically, a few types of virtual machines 
are preconfigured for a user to choose. The lower 
layer is the core layer of scheduling: once user re¬ 
quests are generated, those requests are forwarded 
to next level, which is responsible to choose appro¬ 
priate data centers and physical machines based 
on user requests. CloudSched provides support 
for modeling and simulation of Cloud data cen¬ 
ters, especially allocating virtual machines (con¬ 
sisting of CPU, memory, storage and bandwidth 
etc.) to suitable physical machines. This layer 
can manage a large scale of Cloud data centers 
consisting of thousands of physical machines. Dif¬ 
ferent scheduling algorithms can be applied in dif¬ 
ferent data centers based on customers’ character¬ 
istics. At the bottom layer, there are Cloud re¬ 
sources which include physical machines and vir¬ 
tual machines, both of them consisting of certain 
amount of CPU, memory, storage and bandwidth 
etc. In summary, from the architecture view, the 
compared simulators all adopt the layered archi¬ 
tecture and the layers can be mainly divided into 
three parts. Each layer is responsible for some 
basic functions. At the bottom layer, these sim¬ 
ulators provide management for servers (in both 
GreenCloud and iCanCloud) or hosts of VMs (in 
CloudSim and CloudSched). The upper layer are 
in charge of scheduling the tasks (comparing ef¬ 
ficiency of different algorithms or strategies). At 
the top layer, interface for users are offered, in¬ 
cluding configurations or scenarios that can be set 
by the users in all these simulators. Besides these 
basic functions, some extra functions are extended 
in different simulators, while the basic ones are 
quite similar. 


3. Comparison 2: Building Blocks in Sim¬ 
ulators 

In this section, we discuss the building blocks, 
i.e., the elements modeled in each simulator. 

3.1. Modeling Cloud Data Centers 

In CloudSim and Cloud Analyst, the 
infrastructure-level services related to the 
clouds are simulated by modeling the data center 
entity. In CloudSim, an entity represents an 
instance of a component, like data center or 
host. The data center entity manages a number 
of host entities and these hosts can be assigned 
to one or more VMs based on allocation policy. 
Host represents a physical computing server in 
a Cloud, with processing capability, including 
CPU, memory, storage, etc. In data center, both 
hosts and VMs can be managed during their life 
cycles. 

In GreenCloud, elements are modeled based on 
the multi-tier data center architecture. Servers, 
switches and links, and workloads constitute the 
basic elements of GreenCloud. Servers are re¬ 
sponsible for task execution, quite similar to 
the servers in the CloudSim, and workloads can 
be viewed as the VM requests (tasks) in the 
CloudSim simulator. As for the switches and 
links, they form the interconnection fabric that 
delivers workload to any of the computing servers 
for execution in a timely manner. The VMs are in 
a variety of specification in CloudSim or Cloud¬ 
Sched, while workloads in GreenCloud are divided 
into three types: Computational Intensive Work¬ 
loads, Data Intensive Workloads and Balanced 
Workloads. 

In iCanCloud, the elements model has some dif¬ 
ferences. The main difference lies in the servers 
modeling. In iCanCloud, hardware model repre¬ 
sents the resources provided in the simulator and 
VM instances take the place of servers in other 
simulators. A data center represents a set of Vir¬ 
tual machines, and the VMs are responsible for 
executing the scheduled jobs, which are a list of 
tasks submitted by users. 

In CloudSched, the core hardware infrastruc¬ 
ture related to the Cloud is modeled with a data 
center component for handling VM requests. The 
data center component is mainly composed of 
a set of hosts, which are responsible for man¬ 
aging VMs activity during their life cycles. A 
host is a component that represents a physical 
computing node in a Cloud: it is assigned a 
pre-configured processing capability (expressed in 
computing power in CPU units), memory, band¬ 
width, storage, and a scheduling policy for allo- 
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Table 2: 8 types of virtual machines (VMs) in Amazon 
EC2 


Table 3: 3 types of physical machines (PMs) in Amazon 
EC2 


MEM (GB) 

CPU (units) 

BW(G) 

VM 

1.7 

1 (1 cores X 1 units) 

160 

1-1(1) 

7.5 

4 (2 cores x 2 units) 

850 

1-2(2) 

15.0 

8 (4 cores x 2 units) 

1690 

1-3(3) 

17.1 

6.5 (2 cores x 3.25 units) 

420 

2-1(4) 

34.2 

13 (4 cores x 3.25 units) 

850 

2-2(5) 

68.4 

26 (8 cores x 3.25 units) 

1690 

2-3(6) 

1.7 

5 (2 cores x 2.5 units) 

350 

3-1(7) 

7.0 

20 (8 cores x 2.5 units) 

1690 

3-2(8) 


eating processor cores to virtual machines. A VM 
could be represented in a similar way like the host. 

3.2. Modeling Virtual Machine Allocation 

VM allocation is the process of generating VM 
instances on hosts that match the critical re¬ 
sources, configurations, and requirements of the 
Cloud provider. With virtualization technologies. 
Cloud computing provides flexibility in resource 
allocation. For example, a PM (Physical Machine) 
with two processing cores can host two or more 
VMs on each core concurrently. Only if the total 
used amount of processing power by all VMs on 
a host is not more than available capacity in that 
host, VMs can be allocated. 

Taking the widely used example of Amazon 
EC2 [6], we show that a uniform view of dif¬ 
ferent types of VMs is possible. Table 2 shows 
eight types of virtual machines from Amazon EC2 
online information. The speed per CPU core 
is measured in EC2 Compute Units, being each 
C.U. equivalent to a 1.0-1.2 GHz 2007 Opteron 
or 2007 Xeon processor. We can therefore form 
three types of different PMs (or PM pools) based 
on compute units. In real Cloud data center, 
for example, a physical machine with 2x68.4GB 
memory, 16 cores x 3.25 units, 2 x 1690GB storage 
can be provided. In this or similar way, a uni¬ 
form view of different types of virtual machines is 
possibly formed. This kind of classification pro¬ 
vides a uniform view of virtualized resources for 
heterogeneous virtualization platforms e.g., Xen, 
KVM, VMWare, etc., and brings great benefits 
for virtual machine management and allocation. 
Customers only need selecting suitable types of 
VMs based on their requirements. There are eight 
types of VMs in EC2 as shown in Table 2, where 
MEM is for memory with unit GB, CPU is nor¬ 
malized to unit (each CPU unit is equal to IGhz 
2007 Intel Pentium processor 0). Three types 
of PMs are considered for heterogeneous case as 
shown in Table 3. 


CPU (units) 

MEM(G) 

BW(G) 

Pmin 

Pmax 

16(4 cores x 4 units) 

30 

3380G 

210W 

300W 

52(16 cores x 3.25 units) 

136.8 

3380G 

420W 

600W 

40(16 cores x 2.5 units) 

14 

3380G 

350W 

500W 


CloudSim supports the development of custom 
application service models that can be deployed 
within a VM and its users are required to extend 
the core Cloudlet object for implanting their ap¬ 
plication services. To be exactly, VMs or jobs 
in CloudSim, iCanCloud can only be allocated to 
hosts that have enough resources, like memory, 
storage, etc. 

Workloads in GreenCloud need a complete sat¬ 
isfaction of its two main requirements: computing 
and communicational, which define the amount of 
computing that has to be executed before a given 
deadline and the size of data transfers that must 
be performed prior, during, and after the work¬ 
load execution. 

Currently CloudSched implements dynamic 
load-balancing scheduling algorithms, utilization 
maximization and energy-efficient scheduling al¬ 
gorithms. Other algorithms such as reliability- 
oriented and cost-oriented etc. can be applied as 
well. 

3.3. Modeling Customer Requirements 

CloudSim models the customer requirements by 
deploying VM instances and users can extend the 
core Cloudlet object for implementing their ap¬ 
plication services. The VM instance may require 
some resource such as memory, storage and band¬ 
width on the host to enable its allocation, which 
means assign specific cores of CPU, amount of 
memory and bandwidth to specific VMs. 

GreenCloud models customer requirements by 
configuring the workload arrival rate/pattern to 
the data center following a predefined distribu¬ 
tion (like exponential distribution), or generating 
requests from traces log files. In addition, differ¬ 
ent random distributions can also be configured 
to trigger the time of a workload arrival as well as 
specify the size of the workload. This flexibility 
enables users to adopt various choices to inves¬ 
tigate network conditions, traffic load, and influ¬ 
ences on different switching components. More¬ 
over, the trace-driven workload generation makes 
it more realistic to simulate the workload arrival 
process. 

In iCanCloud, VMs are the building blocks for 
creating cloud systems. Both in the application 
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repository and VMs repository, collections of pre¬ 
defined models can be customized by user. Those 
models will be used in order to configure the cor¬ 
responding jobs that will be executed in a specific 
instance of a VM in the system. Also, new appli¬ 
cation models can be easily added to the system. 

CloudSched models customer requirements by 
randomly generating different types of VMs and 
allocates VMs based on appropriate scheduling 
algorithms in different data centers. The arrival 
process, service time distribution and required ca¬ 
pacity distribution of requests can be generated 
according to random processes. The arrival rate 
of customers’ requests can be controlled. Distri¬ 
bution of different types of VM requirements can 
be set too. A real-time VM request can be repre¬ 
sented in an interval vector: vmID(VM typelD, 
start-time, end-time, requested capacity). For 
example, vml(l, 0, 6, 0.25) shows that the re¬ 
quest ID is 1, virtual machine is of type 1 (cor¬ 
responding to integer 1), start-time is 0 and end- 
time is 6 (here 6 means the end-time is the sixth 
slot). Other requests can be represented in sim¬ 
ilar ways. Fig. 5 shows the life cycles of virtual 
machine allocation in a slotted time window using 
two PMs, where PMt^I hosts vml, vm2 and vm3 
while PM^2 hosts vm4, vm5 and vm6. Notice 
that at any slot, the total capacity constraint of a 
PM has to be met by all VMs allocated on it, and 
each VM has a start-time, end-time constraint. 

In summary, in order to satisfy the flexibility 
and extendibility of customer requirements, these 
simulators all provide predefined configurations as 
well as interfaces for extending. CloudSim can 
extend the core Cloudlet object; GreenCloud can 
generate customer requests in trace log file; iCan- 
Cloud can modify the application model in the 
application and VMs repository; CloudSched can 
change the VM and PM specification in the con¬ 
figuration files. 

4. Comparison 3: Simulation Process 

Generally, the simulation process for cloud data 
centers can be mainly divided into four parts: 1) 
generating customer requests; 2) initiating data 
centers; 3) defining allocation policy; 4) collecting 
and outputing results. The simulators that we 
discussed in this paper all have these four parts, 
though some differences existed when extending 
the basic parts. 

Generating customer requests: Requests 
are generated in this phase and prepared to be 
allocated. In different simulators, the requests 
generation approaches may vary and preparation 
process before requests allocation would also have 



Figure 5: An Example of User Requests and Allocation 


minor differences. Requests in CloudSim, Cloud¬ 
Sched are generated as VM instances and put into 
different queues in different phases, like waiting 
queue represents the requests are waiting to be ex¬ 
ecuted. Workloads are produced in GreenCloud 
with its size satisfying exponential distribution. 
Jobs in iCanCloud can be submitted by user or 
pre-defined model as list and then be added into 
the waiting queue to be executed. 

Initiating data centers: In this phase, data 
center are started to provide resources. The dis¬ 
cussed simulators are almost similar in initial¬ 
izing cloud data centers and they initialize the 
servers/hosts to offer resource like CPU, memory, 
storage and etc. To be noticed, the servers/hosts 
may be geographical separated, which means lo¬ 
cated in different data centers. 

Defining allocation policies: Allocation pol¬ 
icy describes scheduling process, including when 
and how to allocate the specific request to the 
specific server/host. Allocation policy has a tight 
relationship with the goal of scheduling. For in¬ 
stance, load balancing and energy saving may use 
different allocation policies. In CloudSim and 
iCanCloud, First Come First Service (FCFS) pol¬ 
icy is implemented as a basic choice. Cloud¬ 
Sched develops some load balancing policies to 
compare performance and GreenCloud contains 
DVFS (Dynamic Voltage Frequency Scaling) poli¬ 
cies to evaluate energy saving effects. 

Collecting and outputting results: After 
the scheduling process is completed, results would 
be gathered to evaluate the performance of a pol¬ 
icy. Except CloudSim, other simulators would 
present part of simulation results in the user in¬ 
terface. Similarly, with different scheduling goals, 
evaluated indices would vary. The comparison in¬ 
dices and typical outputs would be introduced in 
the following sections. 
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5. Comparison 4: Performance Metrics 

For different objectives of scheduling, there are 
different performance metrics. In this section, 
we discuss some usual metrics that adopted in 
cloud simulators, like for utilization maximiza¬ 
tion, load-balancing, energy-efficient goals. Other 
metrics for different objectives can be extended 
easily based on these usual metrics. Note that the 
four simulator use quite different metrics, here we 
just try to cover the metrics which are applied in 
the four simulators. Table 4 summaries the met¬ 
rics name, metrics objective and the simulators 
that adopt the corresponding metric. 

5.1. Metric for Maximizing Resource Utilization 

In the following, we firstly review two metrics 
for maximizing resource utilization and these 
two metrics are the basis for load balancing and 
energy efficient in the following subsections. 

(1) . Average resource utilization. Average uti¬ 
lization of CPU, memory, hard disk and network 
bandwidth can be computed and an integration 
utilization of all these resources can be used too. 

(2) . The total number of PMs used. It is closely 
related to the average and whole utilization of a 
Cloud data center. 


5.2. Metrics for Multi-dimensional Load- 

Balancing 

In view of advantages and disadvantages 
of existing metrics for resource scheduling 
[5][28][2T][3l], integrated measurement on total 
imbalance level of Cloud data center and each 
server are developed for load-balancing strategy 
[33]. The following parameters are considered: 

(1) . average CPU utilization {CPUY) of a single 
CPU i: For example, if the observed period is one 
minute and CPU utilization is recorded every 10 
seconds, then CPUf^ is the average of six recorded 
values of CPU i. This metric could represent the 
average load on a single CPU during a period of 
observed time. 

(2) . average utilization of all CPUs in a Cloud 
datacenter: Let CPU^ be the total number of 
CPUs of server i, then the average utilization of 
all CPUs on server i is 


CPU^ 


cpul^cpu^ 

E^cpur 


( 1 ) 


where N is the total number of physical servers 
in a Cloud datacenter. Similarly, average utiliza¬ 
tion of memory, network bandwidth of server i, all 
memories and all network bandwidth in a Cloud 
datacenter can be defined as MEM^, NET^^, 


MEM:^^ NET^ respectively. 

(3) . integrated load imbalance value {ILBi) of 
server i: Variance is widely used as a measure of 
how far a set of numbers is spread out from each 
other in statistics. Using variance, an integrated 
load imbalance value {ILBi) of server i is defined 
as: 

{Avgj - CPU^f + {Avgj - MEM^f + {Avgj - NET^f 
3 

( 2 ) 

where 

Avgi = (CPUf -h MEMY + NETY)/3 (3) 

ILBi could be applied to indicate load imbalance 
level comparing utilization of CPU, memory and 
network bandwidth of a single server itself. 

(4) . the imbalance value of all CPUs, memories 
and network bandwidth: Using variance, the im¬ 
balance value of all CPUs in a data center is de¬ 
fined as 

N 

IBLcPu = E^CPUf - CPUA^ (4) 

i 

Similarly, imbalance values of memory {IBL^e^) 
and network bandwidth {IBLnet) can be calcu¬ 
lated. Then total imbalance values of all servers 
in a Cloud datacenter is given by 

N 


(5) . average imbalance value of a physical server i: 
The average imbalance value of a physical server 
i is defined as 

(6) 

where N is the total number of servers. As its 
name suggests, this value can be used to measure 
average imbalance level of all physical servers. 

(6) . average imbalance value of a Cloud data¬ 
center (CDC): The average imbalance value of a 
Cloud datacenter (CDC) is defined as 


IBL 


CDC 

avg 


IBLcpu + IBL mem T dBLjiQi 

N 


( 7 ) 


(7) . average running times: Average running 
time of proceeding same amount of tasks can be 
compared for different scheduling algorithms. 

(8) . makespan: In CloudSched, it is defined as 
the maximum load (or average utilization) on all 
PMs, and in some other simulators, it is defined 
as the longest processing time on all PMs. 

(9) . utilization efficiency: It is defined as (the 
minimum load on any PM) divides (maximum 
load on any PM) in this case. 
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5.3. Metrics for Energy-efficiency 

(1). energy consumption model: 

Most of energy consumption in data centers is 
from computation processing, disk storage, net¬ 
work, and cooling systems. In [5], authors 
proposed a power consumption model for blade 
server: 

U.5+0.2Ucpu+i4:-5e-^)U^e^+0.003Udisk+i3-ie-^)Unet 

( 8 ) 

where Ucpu,Umem,Udisk,Unet are utilization of 
CPU, memory, hard disk and network interface 
respectively. From this formulation, it is observed 
that except CPU, the other factors such as mem¬ 
ory, hard disk and network interface have very 
small impact on total energy consumption. 

In [3], authors found that CPU utilization is 
typically proportional to the overall system load, 
hence proposed a power model as follows: 

P{U) = kPmax + (1 - k)PmaccU (9) 

where Pmax is the maximum power of a server; k 
is the fraction of power when a server is idle, and 
studies show that on average the k is about 0.7; 
and U is the CPU utilization. 

In GreenCloud, Dynamic Voltage/Frequency 
Scaling (DVFS) is considered, the power con¬ 
sumption of an average server can be expressed 
as follows: 


P = Pfixed + ^/ X (10) 

where Pfixed accounts for the portion of the con¬ 
sumed power which does not scale with the op¬ 
erating frequency /, while Pf is a frequency- 
dependent CPU power consumption. 

The energy consumed by a switch and all its 
transceivers can be defined as: 

R 

^switch — Pchassis~^'Rlinecards~^Plinecard~^ ^ ^ Rports^r~\~Pr 

r=0 

( 11 ) 

where Pchassis is related to the power consumed 
by the switch hardware, Punecard is the power 
consumed by any active network line card, Pr 
corresponds to the power consumed by a port 
(transceiver) running at the rate r. 

In real environment, the utilization of the CPU 
may change over time due to the workload vari¬ 
ability. Thus, the CPU utilization is a function 
of time and is represented as u(t). Therefore, the 
total energy consumption by a physical machine 
{Ei) can be defined as an integral of the energy 
consumption function over a period of time as: 

Ei = [ " P{u{t))dt (12) 

Jto 


When the average utilization is adopted, u(t)=u, 
then Ei=P{u){ti — to). 

(2) . The total energy consumption of a Cloud 
data center: The energy consumption is com¬ 
puted as the sum of energy consumed by all PMs: 

n 

Pcdc = '^Pi (13) 

i=l 

It should be noted that the energy consumption 
of all VMs on PMs is included. 

(3) . The total number of PMs used: This is the 
total number of PMs used for the given set of VM 
requests. It is important for energy-efficiency. 

(4) The total power-on time of all PMs used: 
According to the energy consumption equation of 
each PM, the total power-on time is a key factor. 


5.^. C/P (Cost/per task) Metric 
In iCanCloud, in order to deal with the com¬ 
plexity level added by an infrastructure following 
a pay-as-you-go basis, the C/P metric is defined 
as: 


C/P = CT 


ChTexel I Texel 
iN^ ^iN,rr^Nc 


(14) 


where is the task execution time, the val¬ 

ues of / and i correspond to the whole tracing 
interval and the tracing interval per task, that is, 
the grain of the application. On the other hand, 
Nvm and Nc are the number of Virtual Machines 
and number of cores per Virtual Machine, Ch is 
the machine’s usage price per hour. In this way, 
the best infrastructure setup would be that which 
produced the lowest C/P value. 


5.5. Confidence Interval 

Confidence intervals can be calculated for dif¬ 
ferent metrics as follows: Let xi, ^2, X3,..., be 
the calculated metrics (such as IBLtot and Ecdc 
values etc.) from n times of repeated simulations. 
Then the mean is 


^mean 


1 

n 



(15) 


and the standard deviation s is 


5 = 


'Er=i(^ 


- 


n — 1 


(16) 


and the confidence interval at 95% confidence 
(normal distribution) is given by 

il^mean 1*96 ^ 5 :^^mean T 1*96 —) (I^) 

Cn Cn 


9 




Table 4: Metrics Comparison Guideline 


Metrics 

Optimization Objectives 

Simulators 

average resource utilization 

maximizing resource utilization 

All Four 

total number of PMs (hosts) need 

maximizing resource utilization 

All Four 

average CPU utilization 

load balancing 

All 

average utilization of all CPUs in a cloud datacenter 

load balancing 

All Four 

integrated load imbalance value of a server 

load balancing 

CloudSched 

imbalance value of all CPUs 

load balancing 

CloudSched 

average imbalance value a physical server 

load balancing 

CloudSched 

average imbalance value of a Cloud datacenter 

load balancing 

CloudSched 

total simulation time 

All 

All Four 

makespan or longest processing time 

load balancing 

CloudSim, CloudSched 

energy consumption model 

energy-efficiency 

CloudSim, GreenCloud, CloudSched 

total energy consumption of a Cloud data center 

energy-efficiency 

CloudSim, GreenCloud, CloudSched 

total number of PMs used 

energy-efficiency 

CloudSim, GreenCloud, CloudSched 

total power-on time of all PMs 

energy-efficiency 

CloudSim, GreenCloud, CloudSched 

cost / per task 

C / P 

iCanCloud 

confidence interval 

confidence interval 

CloudSched 


6. Comparison 5: Performance Evaluation 

In this section, we will discuss the performance 
comparison of iCanCloud and CloudSim, Cloud- 
Sched and CloudSched with a focus on the scala¬ 
bility. We also compare the typical outputs of all 
compared simulators. 

6.1. Performance Comparison of iCanCloud and 
Cloudsim 

6.1.1. Experimental Environment Settings 

In the comparison between iCanCloud and 
CloudSim, jobs in CloudSim are modeled by con¬ 
figuring input size, processing length and output 
size. The jobs in the simulation experiments have 
5 MB input size, 30MB output size, 1,200,000 MI 
processing length. In addition, jobs would take 
advantage of all the available CPU capacity on 
VMs and the VMs they used are 9,500 MIPS. Of 
course, a new application model is developed in 
iCanCloud to execute the same functionality as 
CloudSim. The experimental environment is on a 
computer with a CPU core i3 and 4GB of RAM 
memory. 

6.1.2. Performance Comparison 

Fig. 6(a) demonstrates the execution time com¬ 
parison of CloudSim and iCanCloud, the x-axis 
presents the number of jobs executed in each ex¬ 
periment, y-axis presents the VMs number and 
its type, and z-axis presents the time required to 
execute each experiment (measured in seconds) in 
log-scale. It’s obvious that both simulators need 
more execution time when increasing the num¬ 
ber of jobs, while these simulators would have dif¬ 
ferent impact when increasing the VMs number. 
When the VMs number is more than 2500, the 
execution time keeps stable in iCanCloud, while 
the execution time is influenced directly by both 


VMs number and jobs number. In most exper¬ 
imental cases with jobs amount less or equal to 
50000, iCanCloud is faster than CloudSim, and 
in all tests with 250k jobs, iCanCloud is faster. 
Under all tests, iCanCloud shows better perfor¬ 
mance in execution time than CloudSim. 

Fig. 6(b) presents the memory consumption 
comparison in each experiment for CloudSim 
and iCanCloud. It can be noticed in this 
graph that iCanCloud requires more memory 
than CloudSim. Up to 1000 VMs, the amount 
of memory required by both simulators is similar. 
However, when using more than 1000 VMs, the 
amount of memory required by iCanCloud goes 
up much faster than CloudSim. 

In general, iCanCloud is faster in large scale 
experiments and provides better scalability, but 
requires more memory than CloudSim. 

6.2. Performance Comparison of CloudSim and 
CloudSched 

6.2.1. Experimental Environment Settings 

In the comparison between CloudSim and 
CloudSched, the comparison is a bit complex than 
the comparison in section 7.1, a new construct 
method is created with start-time and end-time 
parameters, which refers to the lifecycle of a re¬ 
quest. The file size of request represents the re¬ 
quired capacity of all requests. The start-time, 
end-time generation approaches are same, servers 
(named VMs in CloudSim) and requests (named 
cloudlet in CloudSim) both adopt the EC2 spec¬ 
ifications. List Scheduling algorithm is imple¬ 
mented in both simulators, in which requests 
would be allocated to a PM with the lowest uti¬ 
lization. The experimental environment is based 
on a Dell computer with a CPU core i5 and 8GB 
of RAM memory. 
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Figure 6: Performance comparison of CloudSim vs. iCanCloud [4] 
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Figure 7: Performance Comparison of CloudSim vs. CloudSched 
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6.2.2. Performance Comparison 

Fig. 7(a) illustrates the time consumption of 
each experiment, where x-axis shows the requests 
number in each experiment for CloudSim and 
CloudSched, y-axis shows the number of PMs and 
simulators they belong to, and z-axis shows the 
time required in millisecond unit to simulate each 
experiment. It is also apparently observed that 
larger number of requests and number of PMs 
need more time in both simulators. When the 
number of VMs is less than 10,000, CloudSched 
always costs less time to complete simulation. As 
for the numbers of VMs are 50,000 and 25,000, 
CloudSched takes less time than CloudSim, while 
CloudSched takes longer time when the number 
of PMs is more than 5,000. As the ratio of the 
number of VMs to the number of PMs increases, 
like 500,000: 500, CloudSim shows its strength. 
Note that the ratio of VMs to PMs may be vary¬ 
ing from a few to a few tens in a real cloud data 
center. 

Fig. 7(b) shows the memory consumption 
comparison of each simulation in CloudSim and 
CloudSched. In cases when the VMs num¬ 
ber is relative small, like from 1,000 to 10,000. 
CloudSched needs a little more memory, several 
megabytes, to execute simulations. While as 
the requests number becomes larger, CloudSched 
costs much less memory than CloudSim, the large 
difference happens when the request number is 
500,000. The reason is that the VM and PM 
model in CloudSched is simpler than the models 
in CloudSim. 

In general, CloudSched costs less time when the 
ratio of the number of VM requests to the number 
of PMs is not too large (like below 100) and costs 
much less memory than CloudSim. 

6.3. Typical Outputs Compared 

In Fig. 8, we compare the performance of four 
energy-conscious resource management strategies 
against a benchmark technique NPA (NonPower- 
Aware). In the benchmark technique, the pro¬ 
cessors can be operated at higher possible pro¬ 
cessing capacity as 100% and do not consider 
energy-optimization during provisioning of VMs 
to hosts. The first energy-conscious strategy for 
comparison is DVFS enabled, which means that 
the VMs are resized during the simulation based 
on the dynamics of CPU utilization of the host. 
The other strategies are extensions of DVFS pol¬ 
icy: MU (minimum utilization) strategy allocates 
VMs on the minimal utilization nodes; RS (Ran¬ 
dom selection) strategy randomly allocates VMs 
to hosts; MC (maximum correlation) strategy al¬ 
locates VMs on the maximal correlation hosts. 


All these extensive strategies enable the idle nodes 
into sleep mode to save total energy and live mi¬ 
gration of VMs every 5s for adapting to the al¬ 
location. VMs can be migrated to another host, 
if this operation will reduce energy consumption. 
In our simulation, the requests come randomly 
and we vary the number of hosts and VMs to 
obtain data for the energy consumption and the 
number of migrations. From our simulations, MC 
strategy shows the best energy-efficient effects as 
shown in Fig. 8 (b). As for the number of migra¬ 
tions, NPA and DVFS both have no migrations, 
while in other three strategies, MC strategy takes 
least number of migration in most cases. The data 
shown in Figure 8 is the average of 5 times of re¬ 
peated simulation. 
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Figure 9: Typical Output of GreenCloud 


In Fig.9 with GreenCloud simulations, we col¬ 
lect the total energy consumption under variable 
data center load (varying from 0.0, 0.3, 0.6 to 
1 .0) and variable number of servers (varying from 
100 to 400) both for DVFS only and DNS+DVFS 
power management schemes. The x-axis formats 
like (100, 0.0) represents the tests with 100 servers 
and 0.0 load, and (400, 1.0) shows tests with 400 
servers and 1.0 load. In our simulations, we set 
the type of workloads as HPC (High Performance 
Computation) and the results gathered are aver¬ 
aged over 5 runs with the random number gen¬ 
erator. From the bar chart, generally, it’s obvi¬ 
ous that the total energy consumption increases 
as the number of servers increases. It also demon¬ 
strates that the DVFS scheme shows itself little 
sensitive to the input load of servers, while by 
contrast the DNS-^DVFS scheme shows precise 
sensitive to variable load. We also observe that 
under same number of servers and identical loads, 
the DNS-l-DVFS scheme saves more energy than 
DVFS scheme. 

In iCanCloud, Fig. 10 illustrates the results 
gathered by executing the model of Phobos ap- 
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Figure 10: Typical Output of iCanCloud [i] 


plication along with the results of the same ap¬ 
plication implemented on iCanCloud. The figure 
represents the C/P metric for the experiments, 
where the small instance type recommended by 
Amazon EC2 is provided, and the VMs number 
and tracing intervals are varied. From the results, 
we can notice that in some cases, using the same 
size for the interval (in years) and increasing the 
VMs number, causes an upward trend in the C/P 
metric. Then, increasing the VMs number pro¬ 
vides the same execution time, which contributes 
to a increasing of the cost for this configuration. 
Besides that, the mathematical model does not 
represent the time spent on performing I/O oper¬ 
ations. Because that there are still some problems 
for installation of current release of iCanCloud, 
we cannot test more data but using results in its 
original publication. 

In CloudSched, Fig. 11 shows average imbal¬ 
ance level of a cloud data center and five differ¬ 
ent scheduling algorithms for load balancing are 
compared. ZHCJ algorithm introduced in [28] . 
ZHJZ algorithm [21], LIF algorithm [30], Rand 
algorithm, and Round-Robin (Round) are com¬ 
pared. In these simulations, different requests are 
generated as follows: the total numbers of arrivals 
(requests) can be randomly set; all requests fol¬ 
low Poisson arrival process and have exponential 



Figure 11: Typical CloudSched Outputs: Average Imbal¬ 
ance Values of a Cloud Data Center when PMs=100 
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length distribution; the maximum length of re¬ 
quests can be set; for each set of inputs (requests), 
simulations are run six times and all the results 
shown in this paper are the average of the six 
runs. In these simulations, the number of PMs 
is fixed as 100, the number of requests is varying 
from 250 to 1500, and a PC with 2 GHz CPU, 
2 GB memory is used for all simulations. From 
these simulations, we observe that LIF algorithm 
outwits other four algorithms with average imbal¬ 
ance values, which shows that LIF has a better 
load balance effects than others. 

7. Conclusions and Future Work 

In this paper, we mainly compare four open 
source simulators, namely CloudSim, Green- 
Cloud, iCanCloud and CloudSched. These simu¬ 
lators can simulate the cloud data center scenar¬ 
ios from different layers in the cloud computing 
architecture. From their architectures, elements 
modeling, simulation process, performance met¬ 
rics and outputs, we provide detailed comparisons 
about these simulators. Considering the complex¬ 
ity of networks and the difficult to control the 
network traffics, simulators are crucial tools for 
research. We can see that none of them is perfect 
for all aspects and there are still much work to 
do to improve. One suggestion is to use different 
tools or their combinations for different optimiza¬ 
tion objectives such as load balance and energy- 
efficiency. For future work, there are still quite a 
few challenging issues for cloud simulating: 

• Modeling different Cloud layers. As we 

compared in the paper, each tool may focus 
on one layer. Currently there is still lack of 
tools that can model all Cloud layers (laaS, 
PaaS and SaaS). 

• High extensibility. When new policies and 
algorithms are added, modular design of the 
simulators can assure that new modules can 
be easily added, currently the four simulators 
still need improving this. 

• Easy to use and repeatable. The simula¬ 
tors should enable users to set up simulation 
easily and quickly with easy to use graphi¬ 
cal user interfaces and outputs. It can ac¬ 
cept inputs from text files and output to text 
files; can save simulation inputs and outputs 
so that modelers can repeat experiments, en¬ 
suring that repeated simulation yield identi¬ 
cal results. 

• Considering user priority. This is a real 
requirement. Currently the four simulators 


do not consider this yet. Different priority 
policies can be created for users to have dif¬ 
ferent priorities for certain types of VMs, so 
that more realistic scenarios can be consid¬ 
ered. 

• Supporting multiple or federated data 
centers. The simulator should be able to 
reflect and model the multiple or federated 
data centers in real world. CloudAnalyst pro¬ 
vides a framework by extending CloudSim 
and there is still much work to improve. 
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